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ABSTRACT 


This thesis investigates the Selected Acauisition 
Reporting (SAR) System in order to evaluate its effec- 
tiveness. Significant problem areas associated with the 
men and the reporting criteria of DOD Instruction 7000.3 
Werives) are examined. 

The data presented were collected by telephonic survey 
and personal interview with users, reviewers, and 
producers of the SAR. This was augmented by analysis 
of actual SARs for a representative sample of Major Navy 
Acquisition Programs. The major issues examined included: 
timeliness of the SAR, Project Managers' reporting chain of 
command, the SAR review process, cost estimating procedures, 
escaiation and cost growth, role of the SAR, project 
“optimism," rewards and pressures associated with project 
management, Congressional Committee views of the 
SAR, and the background and history of the SAR. 

Although many of these issues require further study, 
proposals are presented to improve the effectiveness of 
the SAR and alleviate some of the controversy that now 


exists concerning the SAR. 
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iN ROOUC TION 


A. GENERAL 

The past decade has seen the acquisition of major 
weapons systems characterized by cost growth in almost 
every major program. Much study has been devoted to the 
cost growth issue and its impact by such organizations 
as the Government Accounting Office (GAO), the Depart- 
ment of Defense (DOD), the Commission on Government 
Procurement, the Blue Ribbon Defense Panel and most re- 
cently the Navy Marine Corps Acquisition Review Council 
(NMARC). 

Current austere defense budgets coupled with severe 
inflation and spiralling weapons systems costs have created 
extreme pressure at all levels of acquisition management 
to check and/or reduce the incidence of weapons system 
cost growth. During the involvement in the Vietnam war 
and to a lesser degree today, members of Gongress have 
found it politically expedient to attack the management 
of defense systems. These attacks serve to intensify 
the pressure at every level of management within the DOD. 
A series of budget, program review, and cost information 
systems have been introduced which were designed to 
improve the weapons system acquisitions process and con- 
trol costs. They include the Development Concept Paper 


(DCP), the Planning, Programming, and Budgeting System 
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(PPBS), and the Selected Acquisition Report (SAR). These 
have had a profound impact on the Defense Procurement 
Establishment. They have provided additional information 
to assist the DOD and/or the Congress in improving the 
acquisition process. This thesis will limit its discussion 
to the SAR system and more specifically to the SAR as it 
applies to and is used for the reporting of Navy weapons 
systems. The author will evaluate the Navy SAR system 

and discuss its effectiveness in presenting the true status 
of the programs. Major problem areas will be identified 

and specific issues raised by the GAO concerning the 

SAR will be discussed. Due to the time constraints imposed 
on this effort, detailed investigation into all problem 
areas was not possible. However, based on the results of 
this study, specific recommendations will be made for 
improvement of the Navy SAR system. Specific questions 


addressed are listed at the end of Chapter II. 
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II. SAR DESCRIPTION AND BACKGROUND 


pee OES CREIPTION 

The SAR is a quarterly report originated by the 
Project Manager (P/M) of each major weapons system pro- 
curement which summarizes the cost, schedule, and per- 
formance data for the program. Major procurements are 
those “requiring a total cumulative financing for Research, 
Development, Test and Evaluation (RDT&E) in excess of 
$50 million or cumulative production investment in excess 


of $200 million.‘ 


Reporting is normally initiated on a 
program after it has been approved by DSARC II and been 
included in the Five Year Defense Program (FYDP). DSARC II 
is the second major decision point in the life of a new 
system. It is conducted by the Secretary of Defense and 

is a comprehensive program review required for authori- 


zation of full scale development of the new weapon. The 


FYDP is the DOD forecast of future expenditures. 


Demedi LOR Y 

The requirement for the SAR was first established by 
the DOD Instruction 7000.3 dated 23 February 1968. Ini- 
tially the SAR was designed and intended only for the use 


of its sponsor, the Assistant Secretary of Defense 
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Boapeparective 5000.1, “Acquisition of Major Defense Systems, ' 
mr vuLy, LOL. 
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(Comptroller). A total of eight DOD programs were included 
in the 1968 experimental phase of the SAR program. 

In the early part of 1969 Secretary of Defense 
Laird articulated the requirement to be regularly advised 
Sree Stawis of major acquisitions. At that time the 
external criticism of defense system acquisition management 
plus growing concern over related cost growth presented 
the need for feedback information which could be compared 
against budget, performance, and schedule plans to high- 
light program status and problem areas. The SAR was 
recognized as a potential vehicle that would fulfill such 
a requirement. The SAR directives were revised to reflect 
the new user and the program moved out of the experimental 
phase, 

During approximately the same time frame GAO and 
Congressional interest in acquisition was intensified as 
a result of hearings on Government procurement, the Anti- 
Ballistic Missile System and the c-5A.° These systems 
were experiencing significant growth in cost estimates. 

tin oemimemlooo, senator otennis, Chairman of the 

senate Armed Services Committee, requested that DOD 
furnish his committee, on a recurring basis, progress data 
on major weapons systems which were in various stages of 
procurement. Additionally, the Deputy Secretary of Defense 
requested that the services nominate additional systems 


DOD Cost Research Symposium Report, 24 March 1970, 


Volume I, Page 4. 


13 





for reporting, using as the criteris, "Those systems 

which might experience technical difficulties and/or 

cost growth. "7 The SAR program and directives were again 
revised to reflect the Congressional user. On 3 September, 
m7, the current DOD Instruction 7000.3 was issued. 

In April, 1972, Change 1 was issued and today that remains 
the official SAR reference document. That change was not 
amajor one. Most of the major changes occurred in the 
1970-71 time frame. 

Since 1972 the Instruction has not been revised; 
however, a large number of memoranda changing the reporting 
format and content have been issued. Currently there are 
40 programs on the list of Congressional SARs. Eight 


other programs submit SARs to the DOD level only. 


feeeoak FURPOSE 
The Secretary of Defense considers the SAR to be, "The 
peenetare DOD source £o be used as a basis for outside 


‘t it is the 


reporting on weapons system acquisition." 
key recurring summary by which DOD reports to the appro- 
priate committees of the Congress on the progress of 
selected major weapons systems. 

MicwomtieuMeory, provides the “textbook” or 
classic example of a management information control 
mechanism. Planning and current estimates are 
presented for quantity, performance, and technical 


eee 


enue Py. 
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characteristics, schedules, and cost. Variance analysis 
and explanations are presented for most areas where de- 
Viations from the plan are encountered. MThis, by 
definition, is the function of a control sub system in 

a "traditional" management system. The feedback loop in 
the SAR system is accomplished when the reviewed SAR is 
Signed, sent to the Congress, and a copy returned to the 
originator (P/M) for use as the base submission of the 


next SAR 90 days later. 


D. SAR CRITICISM 

Despite its potential as the "ideal" management infor- 
mation tool, the SAR system has come under an increasing 
amount of criticism. The Congressional user has questioned 


wae waeccuracy Of the SAR and further criticized it for being 


5 


"untimely. " Some project managers have indicated they 


felt the SAR was a waste of time. Some DOD reviewing 
officials have expressed the view that the SAR was unreal- 
istic. The Commission on Government reported on management 
information systems such as the SAR: 


Congress often cannot act as a credible and sensible 
check on an agency because acquisition programs provide 
no handles to enable Congress to interrelate the purpose 
of the new system and the dollars being spent on them 
with national policies and needs. Instead data is pre- 
sented to Congress in "traditional" forms, inviting 
attention to alresdy defined products and annual a 
incruments that finance development and production. 





Gompsroller General of the U.S., “How to Improve the 
selected Acquisition Reporting System," 27 March, 1975. 


A 


6 “~ Dod ~ 5) mo ” ‘ ~ o “-. m 
Pecermmmeeontracts Report, Number 470, 5 Marsh 1973, Page A-17. 
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GAO applied similar criticism directly to the SAR 
in its 25 March 1975 report to the Congress when it 
questioned why the SAR does not compare system performance 
with the enemy threat and national need. Other Congres=- 
Sional criticism was recorded in the House Appropriations 
Committee report of ll September 1972. The Committee 
noted that the SAR: 
2+. was untimely 
memeread no “audit trail” existing to explain differences 
between development and planning estimates 
»-. has no firm guidelines against which to measure 
additional procurement costs 
»>» has no statement to estimate the probability of the 
weapons system achieving its primary mission or meeting 


momorisinal contract specifications. 


E. QUESTIONS 
Specifically the author will address questions asso- 
ciated with function, concept, and improvements of the SAR. 


meee Docs) tne SAR accurately transmit the true status 
Seeniempmocrzeam:, (Chapter LV) 


mittee rs/chould be the role of the SAR? (Chapter V) 
Seeice the report timely? (Chapter V1) 
- » . Is there too much review? (Chapter VI) 


Peo nomtdetne P/M report directly to the Congress via 
the SAR? (Chapter VI) 


¢ tomowselberm General of the U.S., P. 6. 


Navy, Marine Corps Acquistion Review Committee Report, 
January 1975, Pave VII - 58. 
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. How are the cost data computed? (Chapter VII) 
a. . What is the impact of escalation? (Chapter IX) 
meee. Could the SAR be improved and, if so, how? (Chapter XII) 
The author will attempt to answer these and other 
Pfiportant questions and criticism of the SAR. 
There are other questions that have been raised con- 
cerning the SAR; however, due to the time constraints on 
this research effort, only those issues considered by the 


author to be major ones could be examined. 
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ITit. RESEARCH TECHNIQUE 


A. SAR ANALYSIS 

In order to adequately conduct research into the SAR 
the author found it necessary to visit the Washington D.C. 
area to interview the users, producers, and reviewers of 
the SAR. First he conducted a comprehensive analysis of 
the Navy SARs on file at the Naval Postgraduate School 
(NPS) Library. Next, intensive library research was con- 
ducted to locate and familiarize the researcher with past 
and current literature in the field of Selected Acquisition 
Reports. Although visits to the Stanford, NASA Ames, 
and Naval Postgraduate School Libraries produced no 
literature specifically covering the subject, the library 
material examined did serve to broaden the author’s know- 
ledge in the field of Systems Acquisition. 

Next an extensive telephonic survey was conducted on 
a large number of individuals who were users, reviewers, 
and producers of SARs. ‘This served to provide an effective 
foundation for the ensuing interviews by stimu- 
lating those called to consider the basic ideas of the 
Geseateneprlor tO my arrival. 
B. INTERVIEWS 

The following organizations assisted in the research 
by participating in interviews with the researcher; GAO, 


Office of the Assistant Secretary of Defense (OASD) for 
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Program Analysis and Evaluation (PA&E), OASD (Comptroller), 
Assistant Secretary of the Navy (ASN) for Financial 
Management, Chief of Naval Operations (CNO) OP-96D and 

OP 902, and Chief of Naval Material (MAT-023). Project 
Offices/Project Managers interviewed were: F-14, 

DD-963, LHA, PF, S-3, Lightweight Torpedo, CVAN, A-7, 


Trident, and Harpoon. 
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A. MISCONCPETION 

The author’s initial perception was that the SAR was 
extremely weak and ineffective as a management information 
system. This notion was based mainly on the amount of 
criticism the system had received from various levels of 
the users, producers, and reviewers of the SAR. Part 
of this misconception stemmed from the assumption that 
most Navy Project Managers would be erroneously presenting 
their program in the SAR as one with no problems. 

The initial methodology for testing the effectiveness 
of the SAR was to analyze a representative sample of the 
SARs in the NPS Library to determine how the P/M presented 
the status of his program. The author intended to gather 
data demonstrating that a large number of the major pro- 
grams presented a “completely optimistic" status when, 
in fact, they were experiencing considerable difficulty. 
If historical data, which were available at the time the 
SAR was published, could have been produced which proved 
the existence of project difficulties, even though the 
P/M reported an overly optimistic program status, then 
the point could have been made that the SAR system had 
a major flaw which rendered its effectiveness 
questionable. 

It was logical then to examine programs that had 


experienced known cost, performance, or scnedule 
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difficulties. Before discussing the survey results, it 
Pourdebe helptul ter later analysis to discuss briefly 


pome Otner forces which impact on the status reporting. 


B. P/M REWARDS AND PRESSURES 

The problem of truly accurate status reporting and 
P/M “optimism” was a more complex issue that might 
immediately be apparent. It involved not only the me- 
chanics of executing the implementing DOD Instruction 
concerning SARs but subtly and perhaps more important 
the behaviorial aspects of the rewards, career patterns, 
and measures of success associated with Project Manage- 
ment. As one interviewee stated, "The P/M is rewarded 
morepucting Ships in the water or aircraft in the air. 
He can't give pessimistic estimates in the SAR for fear 
of the program being cancelled."? Another said, "He 
(the P/M) must keep the project alive at all costs."t® 
Most weapons are needed to meet an operational threat 
Sea ecloge' time schedule. Couple that fact with the 
lengthy procedures for initiating a new program plus the 
austere budget climate and you have the genesis of the 
CNO program sponsors pressure on the P/M to "make the 
system work." This pressure plus career enhancement, and 


rewards for delivering the system to the fleet could 


7 Name withheid by request. 
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easily motivate the P/M to present the system status in 
See Opvimistic”’ light. Lourette further showed that 
these kinds of pressure induce the P/M to make program 


decisions that later result in cost srowth. ~~ 


cae |6OPTIMISM" 

The quarterly recurring nature of the SAR gives the 
P/M further opportunity to exercise some "optimism." 
The intense Congressional and DOD concern over cost 
growth pius the nature of the strongly motivated, highly 
dedicated, and enthusiastic officers in P/M billets in 
most cases leads them to exercise their best management 
efforts toward preventing such cost increases. ‘The 
Financial Manager for one program whose SARS recently 
showed a significant unforecasted cost increase stated, 
"The PM knew 18 months ahead of time that the contractor 
was forecasting this increase but he vowed to control the 
Deotrcam 1m such a way as to not let it happen. "+ 

An associated subtle influence on publishing the true 
status of the program deals with the assignment and rota- 
tion of PMs. Some major programs may have a 10-15 year 


acquisition cycle. During that time the average PM may 


di 


Lourette, Richard J., “The Relationship Between Pressure 

on the System Program Director and the Growth of Weapons 

Svewens Cost Hstimates," Boston, Harvard University, 1969. 
1 


Name withheld by request. 
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have a 2-3 year total in the Project Office. These 
memavlons Offer an excellent opportunity for the outgoing 
PM to exercise some of the above mentioned "optimism." 

If there is a problem, it is relatively easy to defer it 
in hopes the new PM will have to handle it. When the 
outgoing PM moves on, his fitness report (i.e. reward) 
normally reflects the known program status at detachment. 
That being the case, the incentive then certainly exists 
for optimism. When the problem later surfaces the in- 
coming PM cannot be held accountable for the problems of 
the previous manager. This was the thrust behind 
Secretary Packards guidance concerning the selection and 
assignment of the pu. Progress has been made in this 
area but further improvement is needed. However, it is 
easy to see that the top performing PM who is motivated 
toward continued promotion could easily be tempted to 
leave the program with the illusion of "no problems." 
Clmment PM retatiton™ plans call for changing at major 
program milestones such as DSARC II or DSARC III (pro- 
duction decision). These major reviews should "clear 


the air" of most major problems and alleviate PM "optimism." 


Dee ENDINGS 
With this foundation the author analyzed the complete 
Nes» cAR files on approximately half of the current list 


of Navy Programs reporting under DOD Instruction 7000.3. 


te Debmmarecetive 5000.1. 
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Pie altnem Ss initial perception about the presentation 
of a no problem status was quickly disproved. There were a 
few examples of "optimism" found. The only sample found 
that supported the author's original perception was: 

"The development program is #~n schedule and within 
budget constraints. "t* 
The following quarter this project reported a significant 
cost growth in the total projected outlay to acquire the 
System. The small incidence of such "optimism" dis- 
proved my initial perception. 

imeracth, the majority of the programs studied pre- 
sented varying degrees of problems ranging from very 
minor to moderately severe. 

For example one major program's SAR reported at 
the 20% completion point as follows; 

Progress reports based on the 12 month or more delay 
in delivery addressed in (a recent SAR)* indicate con- 
tinued slippage in the detail design issues, reducing 
the Navy's level of confidence that the revised schedule 
delivery dates would be met. The contractor has incurred 
an additional 7s months slippage for the first unit to 
12 months slippage for the last unit delivery in (his 
best proposal ).*15 
Concerning the cost estimate on another program, the SAR 
meworucd that the contractor has: 

(1) Expected to go to ceiling, and (2) would propose 
anew escalation clause which would provide escalation 


payments over the full Sens ech Olt period - about one 
year longer than planned.t 


14 


Program name withheld to avoid classification. 


15 *Program name, contractor, and dates deleted to avoid 
elaiscitication. 


16 BID. 
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Even the most casual observer could discern that 
these programs were experiencing difficulties from such 
~AR comments. The programs mentioned are still active 
and currently are experiencing further cost growth. 
Most programs which did experience cost growth and 
schedule difficulties then clearly did not present an 
absence of problems in their reporting. While there 
were other areas that detracted from the effectiveness 
of the SAR, P/M statement of no problems was not one 


Ou onem. 


E. INTERVIEW RESULTS 

Based on the high potential and rewards for expressing 
the program “optimistically” the author did not dismiss 
his initial perception solely on the basis of the analysis 
of SARs. Every interviewer was questioned as to his 
impression as to whether the SAR presented the true status 
of the program. As earlier criticism might suggest, the 
spectrum of answers varied from “extremely accurate" to 
"almost inaccurate" in presenting the true status. Again 
the bias of user/producer was evident; the producers 
generally believing the SAR to be accurate and the user 
calling them ineffective. The SAR reviewers generally believed 
the SAR to be effective. The author concluded that the 
SARS were reasonably accurate with a slight tendency to- 
ward the "optimistic" side. This was based on the extensive 
review procedure for program status; not only the SAR 


review but DSARC and periodic status reviews. These 
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reviews are keyed to revealing problem areas so that, 
while the P/M might be able to cover up cost growth for 

a time, they will eventually be disclosed. Additionally 
the P/M's job could be on the line by the manner in which 
he deals with the Congress. 

One critic stated that the P/M can “tell them (Con- 
gress) what he wants them to know - up to a point." While 
this appeared to be true, the author does not consider 
this to be a major weakness in the system because it gives 
the P/M incentive and opportunity to exercise strong 
control over the potential problem. In the dynamic and 
ever changing acquisition environment many problems 
can be solved without the need of reporting them in the 
SAR. The author submits that reporting every potential 
problem on the SAR would be detrimental to effective 
project management for the following reason: The NMARC 
Report stated that currently the P/M denotes approximately 
30% of his time to briefings, reviews and response to 
questions. *? If he included every problem and potential 
problem the results would not only most likely be exceeding 
the 13 page size limit imposed on the SAR by DOD In- 
struction 7000.3, but more important it would cause a 
significant increase in the amount of time the P/M 
spends in "justifying his existence" because of increased 


program reviews, briefings, and correspondence. 


i NMARC Report, Volume IT, Exhibit III-13. 
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The final point on this issue is that the P/M can 
carry his optimism only so far because of the effective- 
ness of the Congressional Committees in obtaining infor- 
mation on program status from other sources; therefore 
the P/M must be ready to justify his status during 
annual hearings. He can present his program on the 
"optimistic" side but must be prepared to justify his 


optimism. 
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¥Y. ROLE OF THE SAR 


A. GENERAL 

The next area considered was the role of the SAR. 
The question used to trigger dialogue on this subject 
during interviews was: 

PBo you think the SAR should give a financial state- 
ment or balance sheet type status of the project as of 
a date or should it be a forecasting document to predict 
future problem areas?" 

As might be perceived by the reader familiar with 
the acquisition environment the answers were again biased 
depending on whether the interviewee was a user or pro- 


meer of the SAR. 


B. USER DESIRES 

The Congressional Staff users were very polarized 
toward the latter concept. One stated that, "He wanted 
a document of about 5 pages in length that he could refer 
to and use for asking questions during annual hearings." 
He felt that the SAR should be a document that would 
highlight the problem areas in the program for the benefit 
of the Congress. He further expressed the view that the 
DOD was trying to hide information by not making the 
SAR that kind of document. 

Berry and Peckham in their thesis concerning P/M 


dealings with the Congressional Committees made two 
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points relevant to the sar.t® First, the P/M is very 


vulnerable in front of the Congress. They found that a 
P/M could be "fired" if members of the Congress express 
their displeasure to the appropriate people. 

secondly, the Congressional Committees normally cut 
funds from programs rather than add to them. Therefore 
iemimplication is that the best the P/M can do as a 
result of his committee appearance is maintain the pro- 
gram and hope for no cuts. The rational P/M then could 
be expected to present his program only in the status 
that he can reasonably defend during hearings and not 
volunteer any potential problem areas that might cause 
controversey or give reason to reduce the program. The 
earlier discussion concerning the results of presenting 
minor problems and the devastating results in terms of 
intervention by middle and upper layers of management 


applies to this question. 


C. ORIGINATORS CONCEPTS 

On the other hand, the originators (P/M) felt that 
the SAR was similiar to a balance sheet which gave the 
status as of the end of a reporting period. The NMARC 


agreed with this philosophy. Of SAR it said: 


18 Robt. C. Berry and Daniel E. Peckham, Interactions of 


Navy Program Managers with Congressional Committees 
and their Statis, Monterey, Naval Postgraduate School, 
Maren 1975. 
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(ne reports are similiar in concept to the quarterly 
MUEMicsctork telders end creditors 
HOP instruction 7000.3 states that the SARs are: 

: . Standard, comprehensive, summary Status Reports~” 
on major po ease Syeveme for management within the De- 
partment of Defense. 

piece the instruction is vague on this point it 
appears to leave it to the discretion of the individual 
P/M. 

One might argue that a comprehensive, summary status 
report should include every known and anticipated problem 
area. From a theoretical point of view there could be 
considerable support for such an idea. Unfortunately the 
political realities of the acquisition environment make 
auctmce Practice not only difficult to police but completely 
impractical for the same reasons discussed in connection 
with DOD and Congressional involvement and the resulting 
demands on the P/M. Such involvement results in excessive 
time telling them what they want to know and it discourages 
MPiinlabave and Imafinatlon in terms of DOD Directive 


5000.1 with regard to decentralizing authority and 


responsibility for the acquisition of a system. 


47 NMARC Report, Volume I, P. VII-58. 


ae Emphasis Added. 


oh Petmiiemmegion 7000.5 of 13 Sept 71, “Selected Acqui- 
Se oweReports." 
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tee PLIrALLs 

While the Congressional Committee Staffs may promote 
the idea that "if you cleanse your soul and tell us ali 
your problems we'll help you," the realities and their 
record of previous dealings with P/Ms have proven quite 
different as Peckham and Berry found. ©“ The main method 
available to help the P/M involve the area of increasing 
appropriations. The record will show very few instances 
of that and many more examples of funding reductions; 
therefore the P/M would be expected to act in a manner 
which would provide the minimum number of problem areas 


to the Congress during discussions and SAR status reporting. 


Eee FINDINGS 

Respondings to the Congressional user's desires in 
terms of format and content of the SAR might at first 
glance appear to be prudent. However, the author believes 
that the SAR's present role of informing the higher levels 
of DOD and Navy of the program’s status is of vital 
importance. Therefore, the SAR should remain in its 
present format and intent. This idea was expressed by 
Secretary Packard ina letter 9 February, 1970, Letter 
to Senator Stennis where, in discussing the SAR, he stated, 
"Our objective has been to develop a report which fairly 
describes to key executives of the DOD and Congress 


iiemStacus Of OUr acquisitions." 





2G Berry and Peckham. 


S| 





A good reason for retaining the present SAR role 
was expressed by a high ranking Navy Financial Management 
SAR Reviewer who stated that "since the inception of the 
SAR; appropriations sponsors and the OPNAV Staff are much 
eageer versed on the financial status of the program. 
Previously they got “up to speed" only at budget time and 
even then, some of them not so zood."* He further 
indicated that the review process (which will be discussed 
ier) and the very existence of the SAR have served to 
accomplish this increased awareness. ‘The financial, 
technical, and schedule information in the SAR are all 
required to provide and retain this improved state of 
awareness. Based on this discussion and the political 
realities as earlier discussed, the author believes that 
the SAR should retain its current role, that of a balance 
sheet status report. 

It has proven to be quite effective in helping all 
levels of the DOD to be better informed and thus provided 
the potential for more effective management. he question 
of whether or not the higher levels actually do a better 
job because of the information and data in the SAR is 
beyond the scope and time constraints of the authors' 
effort. It would, however, make an interesting and worth- 
while future thesis topic. 

While the author does not wish to prelong the be- 


havorial aspects of the difference in concept of the 





eJ Name withheld. 
Sy 





role of the SAR which exists between the producer and 
the user, he would point out that these steadfast 
opposing views offer some insight into the criticism, 


dislike, and condemnation of the SAR by both groups. 





VI. SAR REVIEW AND TIMELINESS 


The next issue discussed during each interview 
was the SAR review process and the suggestion that the 
P/M submit the report directly to Congress. GAO recom- 
mended such a procedure in a recent report. 

It seems to us that with the emphasis the DOD has 
placed on assigning flag rank officers to manage major 
acquisition programs and with the increased emphasis in 
the past few years placed on developing rewarding careers 
mampro ject Management, it would be reasonable to expect 
wae Project Officer to assume responsibility for the SAR 
content. We recommend that responsibility for the SAR 
preparation be delegated to a single individual ina 
responsible management position. The individual should 
be held responsible for the reasonableness, completeness, 
and accuracy of the SAR. Further, we recommend that the 
Project Officer (P/M)** be delegated this responsibility 
meaminat he certify as to the credibility of the SAR. 

The author was surprised to learn that the majority 
of the Project Managers interviewed stated that they 
preferred retaining the review system as it currently is. 
Their stated reasoning mainly centered on report standard- 
ization, the Review Committee catching errors, and 
matching the budget data with the FYDP, POM, and PDM 
Data. While proof could not be obtained, the author 
perceived that project personnel felt a certain pro- 
tection or shield from pressure due to the fact that the 


Deputy Secretary of Defense currently releases the reports 

DemtCOMoress. tf reporting requirements were revised to 

aH Cia iweatLon added. 

zo Gomourolles General of the U.S., p. 17. 
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make the P/M report directly to Congress this buffer 


would be removed. 


A. RATIONALE 

In considering the levels of review and the review 
Peecess as it currently exists, the over-riding fact 
that must be remembered is that the acquisition team is 
a Military organization and that the P/M is subordinate 
to the Chief of Naval Material, the CNC, and the DOD. 
iempclomote £ood order, discipline, and continuity the 
fiieitary chain of command must be preserved. Lest this 
sound like a "motherhood" statement, consider the impact 
and disorder that could and likely would occur if the 
P/M took every little problem to Congress. More specifi-~ 
cally consider the consequences if for instance OPNAV 
or DOD wasn't giving the P/M all the support he desired 
because of the adverse impact the requested assets would 
have on other programs of equal priority. We have already 
seen that the P/M is rewarded for making his program 
sele@ecca, If his parochial interest led him to reporting 
his dissatisfaction in the SAR directly to Congress and 
he gained Congressional support for the request DOD, 
OPNAV, NAVMAT, and the cognizant Systems Command would 
be forced to respond, probably at the expense cf other 
important programs. Additionally the author perceives 
that the Congress does not desire the role of arbitrator 


to internal disagreements. 





There 1s one Navy example of such direct dealing 
with the Congress. That is the Navy Nuclear Power 
Program headed by Admiral Rickover who is well known 
by the Congress and renowned for his direct dealings 
with them. While it would be difficult to fault the 
results of his program in terms of the hardware acquired 
and the management success acclaimed, the methodology 
has created a Significant amount of stress within the 
Navy acquisition community. 

The timeliness, or lack there of, in the SAR is 
directly related to the review process. SECNAV Instruction 
of 26 February 1972 is the Navy SAR implementing directive 
and gives insight into the review process. The schedule 
for review is included as Table I. The reason for the 
Significant amount of time required for review (currently 
50 days to Congress) is more easily appreciated when 
considering the levels of review, the detail review 
process, and the number of Navy programs currently on 


imewonn L2St: a total of 23 systems. 


B. REVIEW TEAM 

NAVCOMPT, OPNAV, and NAVMAT provided the members of 
the NAW SAR review team. This team met with each Project 
Manager commencing on the 19th day following the “as of 
date" (Table I). Since there was only one review team and 
certain members were required to review every major 
SAR system, the schedule during review week was very 


Tiehte tne autnor learned that each program was alloted 
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(1) 


(2) 


(3) 


(4) 


(5) 


(6) 


a) 


(8) 


(9) 


(10) 


dll, 


rz) 


REVIEW SCHEDULE FOR NAVY PROGRAMS 


Submit advance copies of Congressional (Group I) to 
CNO (OP - 902) no later than close of business on the 
ietimeaverking day after the “as of date." 


Commence NAVCOMPT/OPNAV/CNM joint review of advance 
@epies by 19th working day after "as of date." 


Upon completion of joint review, CNO (Program Sponsor) 
forwards comments to CNM (to include due date of 
Original and final copies). 


NAVSYSCOMS/Pro ject Managers correct/modify reports. 


Originals submitted to CNO via CNM as directed in 
ie above. 


CNO submit originals to NAVCOMPT no later than 5 
days prior to the due date in OSD. 


Eeere talioan Submission to°%OSD in accordance with DOD 
Masthuciion 7000.3. 


Submit non-Congressional (Group II) advance copies 
no later than close of business on the 32nd working 
day after the "as of date." 


Commence NAVCOMPT/OPNAV/CNM joint review of advance 
Gopaes by J3rd work@ing day after “as of date." 


Sep tetinal SARS to OSD no Vater than 45 days after 
tae “as of date." 


ASD (Comptroller) distributes Navy SARs to Assistant 
Secretaries of Defense for I&l, DDR&E, PA&E, and 
Comptroller for review. 


60 days after the "as of date" all final SARs to 
Congress. : 


TABLE 1 


OM, 





me nours for review. At the end of that time if issues 
remained that could not be resolved, they 
heme wcoordinated and staffed with the project officer 
following the review week. The point was that the review 
team had to keep the review schedule going. fMThe priority 
of the review process was to make sure that the SAR data were 
in agreement with einer pudget data such asmche PDM, 
moi; and FYOP. The author was told that in all cases the 
Navy review coordinated changes with the P/M prior to 
executing them. 

At DOD, the review process consisted of routing 
the SARs to those offices listed in Table I rather than 
forming a review team. If one of the offices disagreed 
with information in the SAR they submitted an “issue 
paper" to ASD (Comptroller) who was responsible for 
Poetdimeating and staffing the correction or change. fThe 
ASD (Comptroller) Staff stated that all changes were 
coordinated with Navy reviewers and the cognizant P/M. 
Several project offices reported that this was not always 
the case and that changes had been made without the 
P/Ms knowledge. The author believes that such action was 
most unwise since the P/M must be able to defend the SAR 
during Congressional hearings and didn't always know the 
reason for the change. 

The ultimate releasing authority is the Deputy Sec- 
retary of Defense who signs the SARs for transmittal to 


me 


Comerecae the author found that in the face of all the 





emphasis on informing Congress via the SAR, only 12 


copies of each was sent to "Capitol Hill." 


C. FINDINGS 

[It is the author's opinion that a joint review process 
is essential to the SAR since the Secretary of Defense 
Sonsiders it to be the official DOD source for use as 
the basis for outside reporting on weapons system acquisi- 


26 It is therefore important that SAR data matches 


iron . 
other budget data. The offices represented on the review 
team are the agencies tasked with budget matters and have 
the best qualified people to insure agreement. A fall-out 
of this review process is some semblance of standardization 
among reports. 

The fact that certain key individuals were required 
to review all Navy SARs was found to be a problem and 
one major cause of the length of the review period. While 
the author and several of the project personnel questioned 
had a personal distaste for a review of that length, the 
process does appear to be necessary in order to present 
a unified position to the Congress by insuring that budget 
documents and information reports are in agreement. 

The fact that SAR submission to the Congress is un- 
timely cannot be questioned; i.e., 60 days following the 
close of the quarter and approximately 5 months from the 
beginning of the reporting period. Assistant Secretary 


ae DOD Cost Research Symposym Report, p.3. 


7 





of Defense (Comptroller) McClary in his 18 February 1975 
memo to the various Assistant Secretaries of the Military 
Departments (FM) directed that the 31 march, 1975, SARs 
be submitted to OSD 35 days after the end of the reporting 
period.<° Hhis change is part of the effort to reduce 
the total submission time to the Congress. The current 
@ratt revision to DOD Instruction 7000.3 had not been 
muiblished at this writing but it is anticipated that it 
Vaeeecall for a total submission time of 50 days for 
Congressional SARs; down from the current 60 days. This 
appears to be no more than a token effort at reducing 

the submission time. As stated in the Etat section 
the review process is necessary. The review queue is 
created by the reauirement for a certain group of key 
people to review all programs. Since there are 23 Navy 
SAR programs and each may take up to 2 hours to review, 
the review team is physically time constrained as to 

the number of programs they can and must review. ‘The 
problem is compounded at the OSD level where each ASD 
must review not only all the Navy programs, but also the 


Army and Air Force SARS in approximately one week. 


D. RECOMMENDATION 
The author suggests that if the delay in submitting 


the reports following the "as of" dates is severe enough 


Bl Unpublished ASD (Comptroller) Memorandun, 18 Feb. 1975, 
Moelected Acquisition Reports." 
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in the eyes of the Congressional user to warrant sub- 
stantial change, an alternative approach would be to 
establish a "steady stream" type submission procedure. 
Under this concept each P/M would still report quarterly, 
but instead of having all reports submitted in a group 
at the end of the quarters, one or two programs per week 
would submit a SAR. For instance the F-14 and the 
DD-963 might report the first week, the Harpoon and the 
CVAN the second week, etc. This would remove the review- 
ing queue and reduce the delay in submission. It is 
estimated that the review period could be significantly 
reduced from its present nine weeks. The author believes 
that the total review could be completed and the report 
forwarded to Congress in as little as two-three weeks. 

An associated finding dealt with the apparent 
use of the SAR. Almost universally, the interviewers 
stated that the SAR was not used to drive or make decisions. 
The author could only conclude that this implies the SAR 
was used for background information to complete the total 
picture of program status. Annually the GAO conducted 
a program review and submitted a report to the Congress. 
Additionally the Congressional Committee Staffs reported 
that they had "many contacts." They refused to identify 
the names or types of these contacts. Therefore the SAR 
appeared to be one of the key information sources con- 
Eememio ws prOofram status, but not the only one. This being 


miewcacsce, the actual arrival of the SAR would not seem 





to be time critical except to insure that the current 
Status was available for the annual hearings. 

While this would be a significant change in terms 
of the traditional aspects of the SAR (i.e., conventional 
thinking associated with the end of the quarter, close 
of the fiscal year, etc.) the effect would be to relieve 
congestion at all levels; the reviewer, the producer, and 
the user levels. At the SAR producer level this would 
also relieve the pressure in the NAVAIR and NAVSEA cost 
Givisions. As will be discussed later, the bulk of the 
cost data for most all major projects is originated and 
computed in one of these two centers. 

At the user end of the pipeline, it seems unlikely 
to the autnor that the Congressional Committees are 
able to digest 50 SARs in a single group. This "steady 
stream" submission should not have any detrimental effect 


on Congressional Committee operations or hearings. 


E. CONTINUOUS CHANGE 

One serious preblem area with the SAR system which 
was discovered was the continual change to the reporting 
procedures. In examining the policy changes recorded in 
one of the Project Offices it was observed that since the 
issuance of DOD Instruction 7000.3 in September, 1971 
(ammended by change 1 in April 1972), reporting and 
format change memoranda have been issued on the average 
of once per quarter. The majority of these changes oc- 


Bumeaeta tne pas. year. Recause of this, Preject 
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financial personnei have waited till the quarter was 
completed before commencing their preparation of the SAR. 
This has been a problem in shortening the submission 
time but practically, it has been necessary because of 

the frequent changes. In one case as many as 4 change 
memoranda were issued to the Project Officer between the 
end of the reporting period and the date the SARs were 
finally submitted to the Congress! Each modified the 
guidance of the previous memorandum and two of these were 
dated just one day apart. In that instance, the final 
guidance, requiring another revision to the report was 
issued by the ASD (Comptroller) approximately two weeks 


aiter the reports were to have been due to the Congress. 


F, ORIGIN OF CHANGES 

Many changes are aS a resuit of requests for a par- 
tewlar format or type information from the Congressional 
user. The P/Ms are aware of this. This knowledge plus 
wiempotl tically volatile nature of the SAR fed by all 
the pressure and interest from GAO and Congress has 
resulted in the P/Ms having a dislike for SAR preparation. 
Extreme displeasure results when the P/M has to resubmit 
@Ue te a change in the reporting criteria. It has been 
recognized that the SAR system has cee consider- 


ably and improved since its inception, iiesauuiow. -eund 


that SAR improvement panels/committees had met in the 


28 GAO fetter to Congressman Herbert, 30 Oct. i973. 
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first three years of the life of the SAR and again in 

the Fall of 1974. Considerable effort was expended by 
these groups in gathering the inputs of all users, re- 
viewers, and producers of the SAR concerning improvements 
to the system. The findings were examined by the ASD 
(Comptroller) and other DOD level reviewers and the 
applicable portions included in the subsequent instruc- 
tion and policy revisions. However the continual changes 
to the rules had an adverse effect on the readability, 


E@aeinuity, and understanding of the SAR. 


G. CONTINUITY 

The NMARC panel recognized that refinements and 
guidance from higher authority will cause changes to the 
SAR format: 

However, the timing of such instructions has led to 
additional work (or rework) requirements on the Program 
Offices and the Navy Comptroller's Staff and have re- 
sulted in SAR submissions at dates later than desired. 

This problem is further compounded when one considers 
fiemmpae: On the reader of a report that is in a continual 
State of change. The continuity is broken, the ability 29 
to compare information (and the basis for its presentation) 
from one report to another is Snel le and the creditabil- 
ity of the data is questioned. 

iieweutnor concurs and found that the problem of 
continuity was especially evident during the review of 
entire SAR files. Without the group of policy memos on 


SAR changes it was difficult if not impossible to trace 


a pmippeertilcation Supplied. 


2° NMARC Report, Vol. I, p. VII-61. 
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cost data from one report to the next. Since many changes 
were made as a result of Congressional or GAO pressure, 
the author submits that the efforts of the Congressional 
Beem.) tO Change the report format for their benefit have 
only resulted in making the reports confusing. It should 
be noted that the draft revision to DOD Instruction 7000.3 
was completed in November 1974; however since that time 
two change memos have been issued by the ASD (Comptroller). 
Mhe latest one, issued on 7 March, 1975, will require 
PRomner change to the 7000.3 draft revision. The coor- 
dination of all agencies required to "chop" this draft 
wall result in further delays to the publication on the 
Mew DOD Instruction 7000.3. 

The author strongly recommends that the Congress adopt 
a “hands off" policy to the SAR and let the system function 
to establish continuity and a baseline so that all con- 
cerned can develop an understanding of the system. The 
frustrations with the constant changes were best expressed 


as follows: 


Our point is that we would like to manage with the SAR 
as a useful tool rather than not manage by having lots 
of people changing the system all the time. They (P/M) 
get so Re with that they never have time to. 
manage. 


J Eopuibiiceined Memo, Assistant Secretary of {he Army (FM), 
16 March, 1971. 


M5 





lian 











eco Rao a NiS 


A. BACKGROUND 

While the technical and schedule sections of the SAR 
provide valuable management information, there are few, 
if any, documents that evoke and arouse the emotions of 
the Congressional and DOD Resource Managers as does the 
Beet section of the SAR. The author found a general dis- 
feeecement Of Opinion on the value of point cost estimates. 
In general, it was observed that there was an overall 


jack of knowledge on cost estimation throughout DOD. 


Eee DEFINITIONS 

For the SAR, the cost baseline is either the Flanning 
Estimate (PE) or Development Estimate (DE) deperiding on 
the stage of the procurement. The definition of these 
two terms will be useful to the discussion: 


ingens Esotimate (PE). The estimates of operational 
foemmes! characteristics, schedule and Program Acquisition 
Gost (for both development and procurement) when approval 
is given by the Secretary of Defense for program initia- 
tion. Normally, an approved Development Concept Paper 
erm wiil be used as a source for the characteristics, 
schedule and cost estimates; however in the absence of 

a DCP, a Program Memorandum (PM), a Program Change 

Decision (PCD), Technical Development Plan (TDP) or some 
Similar document may be sued. The specific source docu- 
ment will be identified in the report. The PE for 
procurement will be reported until the initial production 
contract 1s signed. Once a PE baseline is established 

it will not be changed unless specific permission is 
granted by the Assistant Secretary of Defense (Comptroller). 


Development Estimate (DE). The estimates of operational/ 
Werm@omeal Characteristics, scheduie and Proy~am Acquisition 
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Cost for both development and procurement when approval 
is given by the Secretary of Defense for the program to 
move into full scale development. Normally an approved 
DCP will be used as a source for the characteristics, 
schedule, and cost estimates; however, in the absence of 
a DCP, another document may be used and should be iden- 
tified in the report. Development estimate baseline 
figures for procurement may be revised when the initial 
Beoeduction contract 1s signed and should include all 
anticipated options, follow-on effort, etc. Once a DE 
baseline is established it will not be changed unless 
Specific permission is granted by ASD (Comptroller) .32 


ee DECISION POINTS 

The decision points at which the Secretary of Defense 
gives approval for program initiation and full scale 
development are the DSARC I and DSARC II respectively. 

In questioning NAVCOMPT, ASD (PA&E) and project personnel 
the author obtained some interesting observations relating 
to the conflict and controversy surrounding cost base- 
line estimates. 

First, the cost estimates used in DSARC reviews were 
normally derived from the DCP for the specific system. 
The DCP was originated by the military service concerned 
in draft for “comment" and "coordination" within OSD. 

The DCP, after “coordination,” was carried to the DSARC 
and revised as necessary to reflect the modifications 
accomplished during the DSARC. It was then sent to the 
Secretary of Defense for signature at which point it 


consummated a decision. This same procedure now 


J HOD Instruction 7000.3. 





applies at all three major decision points in a weapons 
systems' life.?- 

Interviewees described the actual mechanics of cal- 
culating a cost estimate to be one in which the P/M, 
with the assistance of the cognizant Systems Command 
Cost Analysis Branch, reviewed the contractor cost propo- 
sal and submitted his best estimate of the total cost for 
the system. The methodology universally used among the 
P/Ms was an engineering cost estimate where the work 
breakdown structure was analyzed down to Level Been 
Engineering cost estimating was found to bho vaevery Sout 
area. No Project Office questioned knew of any Navy or 
DOD Instruction covering this type of cost estimating. 
Essentially the P/M and SYSCOM cost shop were heavily 
reliant on the contractor's estimate. This estimate was 
presented in the DCP “draft" and, barring major difference 
from the estimates of the Independent Parameteric Cost 
Analysis Groups of OP-96D and OSD (PA&E), the P/Ms estimate 
emerged from the DSARC as the official forecast of system 
cost in the DCP. The mechanics of cost estimating was 
essentially the same at all @ectcl1 Om seeints. 

DOD Directive 5000.4 of 13 June 1973; established 


the Cost Analysis Improvement Group (CAIG). Their 


33 DOD Instruction 5000.2, 21 January 1975, "The Decision 
Coordinating Paper (DCP) and the Defense Systems Ac- 
quisition Review Council (DS Aine). 


34 DOD Instruction 7000.3. 
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operations have been primarily parameteric estimating 
during review and not the origination of cost estimates. 


Tits Will be discussed later. 


D. PROJECT DYNAMICS 

The author has gone to some length to demonstrate 
how a PE or DE evolved. The reason was to show that 
considerable effort goes into developing such an estimate. 
There are two main problems associated with cost baselines. 

First, all those interviewed agreed that accurate 
eee estimatines 1s extremely difficult, especially in a 
major acquisition which may be six~ten years in length. 
That difficulty can be easily demonstrated by asking any 
home owner to compare the value of the home he paid 
feeOOO for five years ago with its current $60,00-70,000 
price tag. Most weapons systems involve a degree of 
mechnical risk in advancing UhOwmwstate Of tne art.” Add 
that fact to the economic uncertainties of recent years 
emoeune job Of accurate cost estimating for a ten year 
program becomes impossible. 

A related problem involves the very nature of the 
evolution of a new system. The rapidly expanding technology 
in weaponry continually produces new capabilities. Many 
new subsystems can and do evolve during every phase of 
a system. These advances promise to improve its capability 
formcounbering the enemy threat. Ultimately, after some 
of these changes are incorporated and the PE or DE is 


revised to reflect the increased cost, the Uungressional 


ue, 





staffs have interpretated this growth in the price as 
"bad management." 

The Congressional Staff attitude expressed to the 
author was one which almost completely ignored this 
dynamic nature of system evolution. Since actual systems 
cost nearly always exceed estimated cost, one Congressional 
Committee Staff member expressed his displeasure as follows: 
"He (the P/M) came here and asked for $2 million per copy 
for his new system five years ago when asking for approval; 
now today, he asks for $5 million per copy. Thats bad 
management no matter what the system does! You can't 
blame all that on inflation! "?- The impact of inflation 
will be discussed later but here again was the wide con- 
ceptual gap in the users idea of what the SAR was giving 


and the producers thought on what he was submitting. 


E. ESTIMATING DIFFERENCES 

Secondly, the difference in estimating procedures 
caused problems. The P/M submitted an engineering cost 
estimate and the reviewing group used parametric estimating 
techniques. The author perceived that neither group under- 
stood the others estimating technique very well and were 
Somewhat distrustful of the results. While the difference 
in techniques offered some check and balance they made 
mmcditticult to resolve differences in the estimates. 


It was discovered that in most cases, the P/M estimate 


i 
35 Name withheld. 
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had priority and was accepted unless an obvious dis- 
crepancy was discovered and could be proved to the 


heyrewers without doubt. 


F. MOTIVATION 

The author would be remiss if he did not point out 
Wiese Liere exists an excellent potential for contractor 
and P/M “optimism" also in this area. Again, pessimism 
on the part of the P/M would have resulted in added 
pressure and possible reductions to his program or even 
complete cancellation. Remembering that he (the P/M) 
was rewarded for "making the system work" and, further, 
that the contractor which fed him the data was rewarded 
Peeeeoting the contract, the potential for optimistic 
estimates was high. Due to the time constraints on this 
effort the author was not able to examine this question. 
The question of optimism on the part of the P/M and the 
contractor in cost estimates appears to be a fruitful 
area for future research. The author recommends that it 
be adopted as a thesis topic to determine if and to what 
extent the P/Ms and contractors submit "optimistic" cost 
baseline estimates either intentionally or due to improper 


techniques. 


G. SUMMARY 
The author found at all levels a lack of appreciation 
for the difficulty and importance of cost estimating. 


The NMARC Cost Panel concurred with this idea: 


Sul 





For Navy estimates of weapons system cost to become 
fully adequate and have the maximum credibility with 
DOD and Congress, the Navy, from the top down, must 
devote more attention, emphasis, education, and ieretslerer ie 
resources to this subject in each of the SYSCOMs. 


This educational process must be extended on through 
DOD and to the Congress. AS we saw from the Congressional 
Staff member comments and, as will be discussed later, they 
(the Congressional Staff) place extreme importance on 
the difference between the PE or DE and the current 
estimate of the total cost of a system. Their views 
express the opinion that the difference should be "zero!" 
The staff members have argued that both the PE and DE 
should be carried in the SAR through the complete acquisi- 
tion. The author does not concur for these reasons: 
im There is apparent lack of understanding on the 
part of the Congressional Staffs as to the tenta- 
tive nature of the planning estimate and dynamics 
of system evolution. 
2. When both are displayed this would add to the 
size of the SAR and could give an inaccurate 
picture of the program by overemphasizing early 
changes. These early changes are typical, de- 
Sirable and the very reason for having a con- 
ceptual phase - to refine the system. 
3. It would be of little management or analytical 
value since comparing the production model of 
the system to the conceptual model since in most 
cases would be like comparing apples to oranges. 
In most cases they are two vastly different systems. 
H. RECOMMENDATIONS 
iMemccswebaseline should continue in its current 


Status; display the PE for those systems which are 


moported On prior to DSARC JI and the DE only following 


a gee 
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DSARC II. A one time variance analysis should be pro- 
vided for the SAR when the program changes from a PE 
meea DE. ‘This would provide for tracing back te6 the 
projects conceptual phase for those who were interested. 

Due to the dynamic nature of systems acquisition, 

a range cost estimate should be provided in the SAR 
rather than a point estimate. This would highlight 

the truly tenuous nature of cost estimating and relieve 
some of the pressure presently associated witn "cost 
growth." 

The DOD should launch an extensive program to edu- 
cate the Congress and the Defense Acquisition Team of 
Tiemiecire, importance, and difficulty in accurate cost 
estimating and provide the Navy with the assets it needs 
to adequately staff their cost estimating organizations. 

Since engineering cost estimates are so important 
in determining the cost baseline for the program, the 
author was concerned when no policy guidance on the 
subject could be uncovered. In essence, the only check 
on the P/Ms "best estimate": was the parameteric cost 
estimate. Although the author has no evidence that 
errors in engineering cost estimates have been the cause 
more COS) Cverruns;’ the potential for that type of 
mera tion Bee en ret ote. 

eweweor Ol policy OF @uadance concerning engineering 
cost estimates is needed. "Should Cost," which is a DOD engi- 


meering cost estimate offers one alternative to improve this 
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area. Should this be unsuccessful, firmer guidance in 
the form of a handbook or detailed instruction on en- 
gineering cost estimates would be a most unpleasant, 

but apparently necessary step. The author recognizes 
maw this would be an extremely difficult and major step 
in dealing with contractors because the preparation of 
an engineering cost estimate is an internal procedure 
Gueereat complexity. In considering this alternative, 
if the potential benefits outweigh the costs in terms 

of better estimates, reduced Congressional pressure on 
the P/M and DOD, and an enhanced defense management image, 
the handbook or instruction should be published. 

ASD (I&L) Mendolia recognized the need for better 
cost estimating. In an article concerning procurement 
research he said: 

Perhaps the time has come for a guantum jump - to ieap- 
frog the whole inventory of problems that bog down systems 
procurement today - to take the bold step of devising 
something completely new and different. Every time we 
advance the state of the art technically, yesterday's 
equipment becomes obsolete and the pricing experience 
becomes obsolete right along with it. We are continually 
gathering data banks on costs, but they are for yester~ 


day's equipment. What we need is some way tg forecast 
Realistically what tomorrow’s costs will be. 0 


30 Mendolia, Auther I., "How DOD's Procurement Insures Top 
Quality Technolcegy," Commander's Digest, Volume 16, 
Number 19, November 1974. 
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VITL. CURRENT ESTIMATES AND VARIANCE ANALYSIS 


A. GENERAL 

One of the functions of a management information system 
is to compare what was planned with what has been achieved 
Pomedatce, With estimates for completion. In the case 
of complex weapons systems the schedule is fairly easy 
to track. The performance may or may not be easily veri- 
fied depending on the nature of the procurement. For the 
Mree Of Cost, the SAR compares the Current Estimate (CE) 
to the PE or DE and establishes a variance analysis to 
provide some indication of the progress. While this 
appears to be a fairly straight forward precess, the con- 
Peeeration of “what year" dollars, escalation, inflation, 
and which cost estimate to use quickly makes the cost 
analysis a most complex issue. For the purposes of com- 
parison the SAR CE is defined as: 
Current Estimate. Enter the best current estimate of 
cost to buy the service inventory objective. These 
estimates will be based upon guidance given the Project 
Manager by the DOD Component Chief and Secretary of the 
DOD Component. They should be objective assessments of 
program costs. Any evidence of cost change must be in- 
Pierced al the earliest possible date. If the DOD 
Component wishes, the CE may also be reflected in con- 
stant dollars, by footnote. All costs should be escal- 
lated in accordance with established policy on weapons 


ween COSting and inflation factors Uped to compute 
CE should be specifically identified.3 


38 DOD teastruction 7000.3. 





B. CE METHODOLOGY 

The author found that most prejects rely heavily 
On the appropriate SYSCOM Cost Group to either provide 
the basic data to the P/M and his staff or to check the 
data should the P/M opt to prepare his own figures. 

To gain insight into the procedure of preparing 
the CE the author attempted to visit the NAVAIR Cost 
shop. The reply was that they didn't have time. That 
being the case the conclusion that they were understaffed 
would be indicated. The author was allowed to review the 
cost worksheets for the December, 1974, SAR of a major 
NAVAIR Project. An informal survey of several other pro- 
jects revealed that this example was typical of the 
feweedeno2y, Table IT 1s a ficticlous, hypothetical 
example of the worksheet that the Project Qffice would 
use. It is presented only to demonstrate the format and 
methodology. In compiling this worksheet the originator 
Mieiak Costu ocnop) uwses historical cost, existing con- 
maqeus, Contractor estimates, contractor labor agreements, 
earning curve effects, and all factors that are quanti- 
fiable. The ASD (Comptroller) issued the fcllowing 
guidance to the Service Secretaries concerning SAR cost 
estimates: 

The ‘best estimate' should include not only economic 


factors but at §o production factors including the learning 
eurve effect. 


‘d Unpublished ASD (Comptroller) Memorandum, 18 June 1969, 
boelected Acquisition Reports." | 
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Their eStimates are usually calculated in current year 
morlars and passed to the Project Office in a format similar 
to Table II with additional program item lines as appropriate. 
It should be emphasized that Table II is greatly simplified 
tO facilitate its use as an example. The actual worksheet 
examined contained 25 Program Item lines over a 12 year 
period. 

The P/M is then responsible to use these inputs and 
escalate and/or de-escalate the values to whatever base 
year is called for in the existing SAR submission policies. 
The base year for reporting purposes has varied from one 
meporting period to the next. Base years have included 


the current year and the year of program approval. 


C. ESCALATION 

Escalation factors are provided to the P/M by osp. 1 
tajeme Lit 1s the list of current escalation rates as 
specified by ASD (Comptroller) for the various categories 
of procurements. The appropriate economic factor is applied 
memboval for each class of expenditures in a fiscal year. 
The historical and forecast expenditures by fiscal year 
are then totalled to arrive at the P/Ms “best estimate" 
for the total expenditure on the system. 

One set of constraints on the cost section is that 


the P/Ms cost estimates must match the current years 


approved budget and the figures in the FYDP. The author 








+0 Unpublished Memo ASD (Comptroller), 18 February 1975, 
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was told that considerable effort was expended during 
the SAR review process to insure that the SAR estimates 


nemo cOontlict with either of the fore mentioned documents. 


D. VARIANCE ANALYSIS 

Variance analysis in the SAR is provided for all three 
major areas; schedule, performance, and cost. The first 
two areas require nothing more than brief statements con- 
cerning the difference between the CE and the DE or PE. 

The program acquisition cost is quite a different 
poem. HOD Instruction 7000.3 specified that cost should 
be identified separately for development, procurement, and 
construction. Additionally cost variance must be classified 
moe cerms of: Quantity change 

Engineering change 

support change 

schedule change 

Unpredtevable Change (i.e., Act of God) 
Economic change 

Contract performance incentives 
Comiracke COST Overrun 

The author will not discuss each class of variance due 
to the time constraints on this paper. They were presented 
only to give the reader a feel for the complexity the SAR 
has acquired Since its inception. During the previously 
mentioned analysis of the SAR files of major programs the 
author had considerable difficulty in tracking differences 
from one quarter to the next. As an example, a number in 
the variance section would change. No supporting documenta- 


tion or calculations were presented; only an explanation 


such as “change due to estimating refinement." After 
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reviewing a large number of SARs the author was not able 
to gather a great deal of meaningful information from 

this section in each SAR. A considerable number of the 
people interviewed felt that the variance analysis was one 
of the least useful sections of the SAR and none of them 
Said that they used it. Currently the variance analysis 
may consume as much as 30-40% of the pages ina SAR. One 
SAR producer told me that he spends, by far, the largest 
amount of his time in SAR preparation balancing and cor- 
recting the numbers in the variance section. He felt that 
the data were not especially valuable and was somewhat 
frustrated by his perception of uselessness after so much 
work in preparation. 

The current SARs are considerably larger and more 
complex than the first group of SARs submitted in 1969. 
The author had the opportunity to examine the packet of 
initial SARs that were sent to the Congress. Most averaged 
one-two pages total. In comparing the two generations of 
the SAR the immediate impression was that current SARs 


are far too complex. 


E. RECOMMENDATION 

The author believes that due to the difficulty in 
following program variance analysis and its apparent 
marginal usefulness the variance analysis sections of the 
SAR should be deleted. This would simplify the reports 
and improve the readability with little loss of ability 


to present the status of the program. 
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IX. ESCALATION AND COST GROWTH 


A. GENERAL 

The study of escalation and cost growth is extensive 
ewouen tO provide the material for more than one thesis. 
The author was not able to investigate the subjects in 
detail and will only briefly introduce and discuss the 
problems to demonstrate how volatile the SAR becomes as 


a result of the differences of opinion on these two issues. 


B. ESCALATION 

As mentioned in the previous chapter the P/M applied 
miemesccalation factors supplied by the ASD (Comptroller) 
to the cost data breakdown by year to derive the totai 
beagemanm COSt. Obviously, the factors applied could change 
the total program cost by a Significant amount. To empha- 
page, Onis point during periods when actual inflation rates 
exceeded DOD projections a sensitivity analysis was added 
to the total program cost section of the SAR which pro- 
toeemomcost Giiferential factor to add or subtract from 
the total program cost estimate to adjust for a different 
escalation rate. 

An example of this type of impact was observed by the 
author during the SAR analysis phase of this research. 
A difference of only 2% in the escalation index resulted 
in a $50 million cost increment to acquire the target 


inventory for a system whose estimated total. cost was 
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slightly in excess of $1 billion. It can be easily seen 
then that a small forecasting error during the unexpected 
"double digit inflation" of 1974 resulted in devastating 
growth to systems costs as will be discussed in the "Pre- 
vious Rates" and "Impact of Realism" sections of this 


chapter. 


C. INFLATION AND ESCALATION | 

Another "soft" area in the SAR was the entire concept 
of escalation and inflation. The enclosure to DOD Instruc- 
tion 7000.3 (Paragraph II E3) seemed to imply that the 
two terms were interchangable. In discussing the procedures 
for computing the current estimate it said: 

All costs should be escalated in accordance with es- 
tablished policy on weapons system costing and inflation 
factors used to compute CE should be specifically identified. 
During many of the interviews the author asked for an 
explanation of the difference of the two. Most interviewees 
stated that they didn't know what or if there was distinc- 
tion. One felt that escalation could be divided into 
economic and non-economic escalation. Non-economic esca- 
lation was cost growth associated with items such as 
change orders and quantity adjustments. He stated that 
economic escalation was attributed to factors such as 
inflation or price index increases. 

Another interviewee said that escalation was any cost 


meow.  knrtation, in his view, was that portion of 


escalation due to the increase in price levels; l1.e., the 
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change in the cost of a "Standard market basket" of con- 
sumer goods or increase in some price index. 

The author contends that the two terms are intended 
wemec interchangable for the purposes of the SAR. In the 
SAR variance analysis, inflation was in the economic change 
category (page 69). However economic change in enclosure 
to DOD 7000.3 (paragraph II G4) was defined: 

Economic Change. A change due to the operation of one or 
More factors of the economy. This includes specific 
contract changes related to economic escalation and the 
economic impact portion of quantity changes not accounted 
for by the original cost-quantity relationships used to 
calculate cost-quantity change variance. Constant or 
current dollar amounts in program estimates to reflect 
() altered price levels and (2) definitized contract 
amounts. 

This subject provides another fruitful thesis research 
area. The author was time limited but felt that a brief 
introduction would be beneficial in highlighting the lack 
of understanding in this area during economic fluctuations. 
inflation has been such a provocative and emotional sub- 
jgect that no matter how it was handled in the SAR or any 
other reporting system there would be controversy gen- 
erated over the methodology and results. The SAR users 
feomeresclonal Commattees) have been particularly critical 
of inflation in the SAR. They felt the P/M was trying to 
blame all cost growth on inflation. The staffs felt 
that inflation was not solely to blame. Since the exact 
inflation impact could not be identified in the SAR, the 


staff members were very suspicious of the validity of the 


Sosu data. 
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Mhile the author could obtain no direct evidence, 
there was a strong presumption that this suspicion on 
tie part of the Congressional Staffs led to pressure on 
DOD which culminated in the 7 March 1975, ASD Comptroller 
Memorandum requiring two SAR cost formats. One estimate 
will be in current year dollars. The other in base year 
dollars; with the base year, in most cases, the year the 
project was first authorized. The apparent intent was 
fmaty it the P/M de-escalates all costs (including the CE) 
back to the base year, the difference between the DE and 
CE in those same year dollars would be the actual cost 
growth not attributable to inflation. While on the surface 
the concept seems to be sound, the familiar questions of - in 
what year dollars do you price a contract change, what 
deflation factors whould be used, and which cost baseline 
again become extremely important. As we will see in the 
next few sections of this chapter, the compounding effects 
of a small escalation/de-escalation rate difference lead 
tO Significant dollar differences over the 8-15 life span 
of most SAR projects. The author questions whether any 
meaningful data will be produced with the two formats. 
A phone check with two Project Offices during the final 
stages of the writing of this thesis has revealed that 
there 1s considerable disagreement over the above questions. 

Further, an extraordinary amount of time has gone into 
ime preparavion of the extra format. The concept of the 


eeaicevalieo or Information schould be considered. The 
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author recommends a study to evaluate the use and worth of 
this information. It is possible that the Congressional 
Committee Staffs, in an attempt to fulfill their per- 
cieved role of reducing budget requests to give the il- 
lusion of control (Berry and Peckham, p.57), have gone 

too far in reviewing past data attempting to highlight 
"bad management." While those historical data are valuable 
in making program decisions, the main emphasis in program 
management and review should be the future or completion 
of the project. If the decision is reached to continue 

a program the efforts should be concentrated on optimizing 
the remainder of the program rather than focusing on the 
historical data. Historical data should be a means and 


mob an end. 


D. CONTRACTOR ESTIMATES 

several project personnel were questioned as to 
whether or not they felt the contractor was "inflating" 
the cost estimates he provided. The majority response 
indicated they didn't know or weren't sure. This area 
was one in which the author recommends immediate investi- 
eaohom. Lt Che contractor 1S “inflating" his cost 
estimates for future portions of the program and the P/M 
later applies an escalation factor the resulting cost 


eoulmave 15 in error. 


Poe Re VIOUS RATES 
Prior to the June 1974 SARs the maximum escalation 


rate permitted by DOD was 4.5% annually. The 4.5% rate 
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was for FY 1975 followed by 3.1% for FY 1976 and beyond. 
These indices were maintained despite the fact that the 
economy as a whole was experiencing an inflation rate 
far in excess of that. For example: 

In December (1974) the all commodities WPI (Wholesale 
Price bas) was at 171.5, 20.9% higher than a year 


earlier. 


imebecember (1973), the all commodities, WPI was 145.3, 
18.2% higher than it was a year earlier. 


ime September (1974) CPI (Consumer Price Index) was 
12.1% higher than a year ago.‘ 


These factors were well above authorized DOD estimates. 
As an example the WPI index for the base year of 1967 
provided an index value of 100. When the author compared 
the 1974 WPI index number 171.5 with a standard escalation/ 
de-escalation chart provided as Table IV for a 7 year 
period (1967-1974) the average escalation rate factor was 
8%; approximately two to three times what had been used 
in previous SAR cost estimates. Even using the CPI 
number 151.9 from Table V gives an average yearly esca- 
lation rate above 6%. 

The author intends to merely introduce this subject 
but as can be readily seen it provides a fruitful area 
for further research into its application to the SAR. 


+ 
41 Emphasis added. U.S. Department of Labor, "Wholesale 


Prices and Price Indexes," December 1974, P. l. 


42 


Opole partnent O1 Labor, “Wholesale Prices and Price 
iigexes a December, 1973, P. 1. 


go U.S. Department of Labor, “Consumer Price Index De- 
ieltedeneport,’ september. 1974, P. 1. 
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GNP DEFLATOR 


(1958=100) 117.6¢+ 172.19 
WPT 

(1967=100) 100 171.5 
CPT 

(1967=100) 100 151.9 


INFLATION INDICIES 1967-1974 
TABLE V 


A thorough investigation of this aspect of the SAR 
system, wnile eminently worthwhile, is beyond the scope 
eens study. 

The reason for these lower rates of escalation could 
not be actually determined and can only be speculated upon 
by the author. It appeared to be a political constraint 
imposed for the reason that a major government agency, 
such as DOD, could not be permitted to forecast higher 
mbeseOt Initiation without the prospect of contractors, 
labor unions, and others responding in such a way as 
to making them come to pass. While this may have proved 
politically expedient the result was an extraordinary 
increase in cost estimates when more realistic values 


Ah 


U-s. Department of Commerce, Statistica) Abstract of 
miemUroe io + 0p. 404. 


¢o U.S. Department of Commerce, "Survey of Current Business," 
Vom l2,) December 1974, p. s-2. 
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were used. The large cost growths associated with such 


reassessment resulted in an outcry from the Congress. 


F, IMPACT OF REALISM 

When more realistic rates were applied such as 11% 
for FY 1975, 8% in 1976, decreasing to 4.3% in 1980 and 
3.7% annually thereafter, the grand total cost estimate 
for 44 major DOD acquisitions rose to a total of $16.9 
peeiion; trom approximately $90 billion to $106 billion. 
Of that increase Navy programs were responsible for $7.3 
mtiton from a baseline of $50 billion. In commenting 
on the increases ASD (PA&E) Sullivan re-affirmed his 
Soriiton of the lack of understanding on Capitol Hill as 
to the tentative nature of cost estimates. He forecast 
that the Pentagon will "be told by some members of Congress 
that that's how bad we managed between the first and second 
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Glarters ." His acon eee was born out as evidenced by 
the increased number of inquiries by Congressmen, Senators, 
and their staffs. A SAR Improvement Group was established 
within DOD to try to ease some of the pressure. The GAO 
conducted another study of the SAR. Since that time 
several new change memoranda for DOD Instruction 7000.3 
have emerged. A draft revision to the same instruction 
meemerenuly in routing for coordination and publication. 
While it would be difficult to tie each of these directly 


n° Wierosvacembally,” Vol. 69 #22, 2 October 1974, p. 169. 
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to the revised escalation indices the fact remains that 
they followed shortly after the public announcement 
concerning the new cost estimates. 

Although the rates indicated in Table III may have 
come close to what actually happened for what is now 
known of the 1974-1975 period, the reader might challenge 
Piewtates indicated for future years. Speculation about 
the political constraints, again in terms of limiting 
those indicies, could be well founded. The political 
meoolem sot DOD “officially” embracing future inflation 
rates considerably higher than those then being experienced 
waS a very sticky one. More significant was the economic 
uncertainty in general. That subject will be discussed 


in the succeeding chapter. 


G. COST GROWTH 

Cost growth, in simple arithmetic terms, was the 
difference between the current estimate and the baseline 
estimate. As has been previously discussed, both elements 
of the cost growth equation (CE and DE) were very "soft" 
and subject to ranges depending on what escalation indicies 
were used. The engineering cost estimating problems 
increase the uncertainty as to what the actual cost 
growth was. 

This cost growth was also extremely difficult to trace 
meomeone Gquartvterly report to the next because of the 
EaoLdlyechangsing format and guidance in computing estimates. 


Many readers of the SAR were not on the routing for new 


(iis 





change memoranda issued by ASD (Comptroller) and therefore 
had no knowledge of the many changes. As the author 

found during his own investigation, without the current 
guidance and all the changes in between, it was nearly 
miopossible to refer to DOD Instruction 7000.3 and trace 
the cost growth. 

In essence.then, the current guidance had a significant 
impact on the current cost status of the program in the 
~AR. The total program cost estimate could vary over a 
wide range, depending on the projected escalation. Due 
to the dynamic nature of weapons system evolution, the 
baseline, in many cases, was not an accurate reflection 
of the true system capabilities and components. As we Saw, 
during development it was subject to many changes. This 
is not to say that it shouldn't have changed, merely that 
imomerdeand this made it extremely difficult to put “handles" 
on the process with which the Congress could evaluate the 


P/M and his management performance. 


Ge 





X. ENVIRONMENT 


A brief description of the environment in which the 
SAR and the P/M must live and function would be beneficial 


to the reader prior to summarizing the findings. 


foe CONGRESS 

The U.S. budget in the past several years has under- 
gone a transition in that Defense no longer gets the 
largest share of the annual budget. "In the current 
Federal budget 70% of the dollar expenditures are uncon- 
prediiable."*¢ Most Defense acquisitions fall into the 
30% that is controllable. Since considerable Congressional 
review occurs prior to appropriating these monies, the 
SAR has emerged aS a primary information tool to transmit 
the status of per euler programs to the Congress. Berry 
and Peckham reported that the Congress sees their role 
as one of cutting the funds requested for some programs to 
ensure that the illusion of control is projected. In this 
light many Congressionally Requested Changes to the SAR 
format and content appear designed to make spotting problems 
and cost growth easier. This was the apparent motive 
behind the 7 March 1975 ASD (Comptroller) change memo- 
randum to the SAR which called for de-escalating all costs 


and estimates back to the program base year. ‘They felt 


a Unpublished speech by Senator Barry Goldwater, 11 April 
1975 to the National Student Symposium on the Study of 
the Presidency. 
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Biaeethen the Current Estimates could be compared to 
Development Estimates or Planning Estimates in the same 
year dollars for a simple calculation of cost growth. 

The Congressional proponents of this methodology base 
their analysis and program evaluasion on two estimates. 
They had little appreciation for the tentative nature of 
those estimates or the dynamics of project evolution. 

As was previously shown, the planning (escalation) factors 
determined to a large extent the cost estimate for com- 


pletion of the program. 


B. CONTRACTORS 

The contractors in many defense industries are no 
longer as dependent on DOD business as they once were. 
This is especially true in the Navy's unigue area of 
shipbuilding. The past ten years has seen a significant 
decline in the number of American shipbuilders. The ship- 
building industry has long suffered the inefficiencies 
associated with the complexity of constructing a state 
of the art "man-of-war." The shipyards currently have a 
large backlog of orders for oil tankers and ore carriers. 
These vessels are much less complicated to build than 
combatant ships. Additionally, commercial business does 
MooecOMmmib the contractor to costly and complicated 
Management assistance programs such as Government Quality 
mi Wieance, Military standards/Specifications, and CSCSC 


(DOD Instruction 7000.2 series). Conseauently the Navy 
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mee having difficulty in obtaining competition for some of 
imesemajor Contractors. In this less than competitive 
environment there is not as much motivation for accurate 
cost estimates; especially in the cost-plus type contracting 
Situation. There are special problems in shipbuilding that 
do not exist in the aircraft or missile programs. Fore- 
most is the time element. The complexity in shipbuilding 
wOumbaemMake cOSt estimating most difficult in a "sterile" 
environment free from inflation over a 10-15 year period 
required for completion. It 1s impossible when a new 
shipbuilding program pushes the state of the art. Add to 
that the violent economic conditions of the past two years 
coupled with shortages of basic raw materials and the ever 
escalating demands by labor and the contractors, and a point 
pPowimeate Of what the Number 7 Trident Submarine will cost 
6-8 years from now is very "soft." These appear to be 


the ingredients for guaranteed cost growth. 


C. CORPORATE PLANNING 

To amplify the plight of the P/M in dealing with the 
eomueac vor GUring Per ods of economic uncertainty, major 
changes have apparently transpired in the past year, 
especially in the area of long range corporate planning. 
Many major companies (including some major defense con- 
fMeletore) have forecast continuing high inflation (con- 
iar LOrto) (Comptroller) guidance), continuing tight 


money, and high long term interest rates. Some are 


ie 





revising their planning to include the "worst case" which 
in certain instances is a full depression. This has been 
are rPicult due to the opposing, long standing traditional 
economic model built on a straight course of optimism. 
several quotes from a recent special report on this sub- 
ject are worth reviewing: 


As risks mount, companies routinely demand of their 
businesses a higher return on investment. 


In line with trying to reduce the risk exposure in 
the company, and at the same time not lose any of the 
entreprenenrship, we (General Electric, a major DOD 
contractor) have moved toward more organically grown, 
smaller-in-size, larger-in-number ventures. 

iitewepredicting the future worth of the dollar is a 
Major planning headache. American Standard (Corporation) 
encourages managers to translate dollars in forward 
plans into ounces of gold, not only to keep them aware 
Semene COntinuing erosion of the dollar but to dramatize 
the need to look at market forecasts in terms of physical 
units oo than paper money, which overstates market 
growth. 
These quotes were mentioned to emphasize the wide span 
between the Government and the business world on views 


Oieene future. 


D. GOVERNMENT PLANNING 

The DOD guidance on economic escalation of 3.1% 
beyond 1980 couldn't be classified as "continuing high 
inflation." The concept of "smaller-in-size, larger-in- 
number" ventures can't refer to major acquisition programs; 
which means General Electric may become less interested in 
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"Corporate Planning-Piercing Future Fog in the Executive 
suite," Business Week, 28 April 1975, p. 5! 
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DOD contracts. The Armed Services Procurement Regulation 
(ASPR) and the Report of the Commission on Government 
Procurement have indicated a trend for placing more, not 
fe-ec, Fisk On the contractor. As for the contractors 
higher return on investment, the audit agencies of each 
service, tne DOD, and the GAO are all growing in size 

and importance. They carefully review contractor records 
womverity all cost data. There is a contract renegotiation 
beata tO Ensure that a contractor's profits are not 


Se@essive. 


eee BoTIMATE OF SAR IMPACT 


What is the impact in Congress when a program reports 
peetemificant cost growth in its SAR? While no firm data 
could be gathered on this question, the discussion was 
deferred to this section because the environment has a 
Significant impact af the author's answer to this question. 

fimoGeune ok 1S NOt a decision making document. The 
interviewees almost unanimously agreed that no decisions 
are made based on the information in the SAR. The reasons 
cited were that the SAR was not timely enough and that 
the PPBS system plus the annual budget hearings provide 
the procedures for the majority of the decisions concerning 
a program. Remembering that the SAR's mission was to 
inform, the purpose of reporting cost growth would merely 


be to highlight this information to the Congress. 
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The recent course of events when a program suffers 
cost growth has been to reduce the numbers of the system 
bought and/or lengthen the program cycle in hopes of 
Meacer approval of additional funds. It would be difficult 
to forecast that pattern changing, unless it became more 


restrictive! 


F, OUTLOOK FOR THE SAR 

imeetnie mids of these problem areas, the 
P/M will be under more pressure than ever to reduce 
cost growth and control cost. The SAR will become more 
prominent as the Congress and its Committees expand their 
base of direct control over the procurement team. The 
author sadly forecasts more, not fewer, changes to the 
SAR format and reporting procedures as the Congress strives 
to gain the information they perceive is required to 
Seereise this control. 

Until improved appreciation is gained at all levels 
oe tie benuous Nature and difficulty of accurate cost 
estimating for a ten year program in an uncertain environ- 
ment the SAR will continue to exist in a cloud of contro- 


versy and continual change. 
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XI. FINDINGS AND CONCLUSIONS 


A. FINDINGS 

The author found that there was considerable disa- 
greement on a number of the issues including the role 
of the SAR, the review process, the question of accurate 
Serius reporting, and the definitions of escalation and 
Gintfiation. In general there was little appreciation at 
the user level for either the dynamic nature of a major 
acquisition or the tenuous nature and difficulty of 
point cost estimates. 

The queue for the review process was created by the 
fact that a small group of people essentially review all 
Navy SARs; they are physically time constrained from 
speeding the process to any great degree. An associated 
problem was the frequently changed reporting guidance 
which constrained the P/M to waiting until the "as of 
date" to commence SAR preparation thus contributing to 
the length of the review process. The SARS are untimely 
in terms of their arrival at the Congress. This appeared 
not to be a significant factor since it was discovered 
that the SAR does not drive any program decisions. 

femtie P/M level, the incentive for “optimism" exists 
in terms of career rewards. Very few projects present 
Hieteestatuceas ‘complete Optimism"; i1.e€., only minor 


or no problems. ‘The P/M used engineering cost estimating 
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techniques for calculating baseline cost estimates. Slight 
differences in escalation rates used for cost estimating 
could produce large changes in the estimated price for 
most systems. Until recently, unrealistic escalation 

rates produced unrealistic low estimates. More significant 
was the large impact of applying more realistic escalation 
rates to program costs. In general, the P/M and his staff 
had a dislike for the SAR due to the large amount of time 
required to prepare, review, revise, and answer questions 
concerning the SAR. 

Overall, the SAR has changed considerably since its 
inception. The original Congressional SARS were a maximum 
of two pages each. Now the SAR has grown so much that a 
13 page limit was imposed. It has also increased in com- 
plexity and now includes cost, schedule, and performance 
data plus a variance analysis and contractor information 


sections. 


B. CONCLUSIONS 

The author concluded that, due to the problems of 
varying escalation rates, which compound cost growth, 
questionable cost estimates, and the dynamic nature of 
system evolution, the SAR was only reasonably effective 
mimeceurabchiy presenting the cost status of a project. 
The amount of review seemed to insure that it was accurate 
for the present but that it may not show all the problems. 
In general, the performance and schedule information 


wis accuya te. 
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There is considerable room for improvement of the SAR 
system, especially, in terms of cost estimating and its 
understanding, SAR timeliness, frequency of revision of 
reporting guidance, variance analysis, and overall com- 
plexity. The succeeding section will list some specific 
recommendations for improvement. 

The area of cost data computation was a very “soft" 
one. Since cost data for the SARS were computed or moni- 
tored by the appropriate SYSCOM cost shop, there should 
be firm guidance in terms of standardized estimating 
procedures. 

The SARS arrival at their ultimate user, the Congress, 
could not be termed timely (60 days following "as of" date). 
This issue was tied directly to the review process. The 
author concluded that it would be very difficult to 
expedite SAR submission under the current practice of 
submitting them in a group. 

The review process was needed to provide some degree 
of standardization among programs and ensure that SAR 
data were in agreement with FYDP and budget data. In 
addition, without the file of change memoranda it was 
difficult if not impossible to track changes in the SARs 
of any program. 

The author concluded that the SAR should retain its 
present role of providing the status as of a date rather 
rather than being an expanded "program highlights" 


document only, highlighting problem areas. This was 
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based on the important function the SAR was filling; pro- 
Viding high ranking Navy and DOD officials with data 
necessary to improve management of the several programs. 
if is not to say that major problems should not be 

noted in the SAR; however the inclusion of every potential 
problem would introduce many middle layers of management 
and suppress P/M initiative and aggressive management 
techniques. 

In the same light, the P/M should not report directly 
to Congress with the SAR. The fact that the procurement 
team is a military organization makes it necessary that 
the superiors of the P/M (Navy and DOD) review the inputs 
to ensure concurrence with budget data and the established 
priorities of the entire Defense establishment. 

The author concluded that point cost estimates are 
Emeost worthless in the uncertain economic conditions, 
Hong lived programs, and the rapidly changing inflation 
that have recently been experienced. 

Overall the author believes the SAR is too complex and 
time consuming in its preparation. The author concluded 
ici tie Concept of the SAR or any Similar information 
document is a sound one; however, the SAR user must 
recognize the inherent limitations on comparing two 
estimates to provide an exact measure of cost growth and 
management evaluation. The author concluded that the 
economic uncertainties and technological advances during 


a long term major acquisition will result in changes and 
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cost growth. Attention should be focused on those 
decisions concerning the future of the program. Since 
the magnitudes of the estimate to complete is a direct 
function of the escalation indicies the author would be 
most interested in the rates the Congress would recommend 
for use. 

The NPS Library needs to be expanded to include more 
procurement literature since the System Acquisition 
Curriculum has received more attention and interest at 
high levels. Specifically the "Aerospace Daily" Newspaper 
should be ordered. Most important is an updated file of 
DOD and Navy Instructions which are invaluable reference 
material for student research. 

Many of the areas examined and some of those not 
examined in this thesis require further research. Specific 


recommendations are included in the succeeding chapter. 
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XII. RECOMMENDATIONS 


itiemoak should retain its current role; an “as of 

date" status and not strictly a problem highlights 

eee UME Mb . 

The P/M should not report directly to Congress with 

the SAR. The concept of review should be retained to 
ensure POM, PDM, FYDP, and budget data are in agreement. 
The Congress should consider the issue of SAR timeliness. 
If the current system is unsatisfactory due to late 
SAR arrival the alternative of "steady stream" sub- 
mission is recommended. This would still involve 
emery G@eporting for all projects but instead of 
submitting at the quarters end, approximately two 
reports per week would be submitted and reviewed. 

The Congress should develop a "hands-off" policy with 
respect to the SAR. The continual requests to DOD for 
SAR format and content changes has destroyed the 
continuity in the reports, made them difficult to pre- 
paGemendmrcedsmcpecitically the 7 March 1975 ASD 
(Comptroller) Memorandum requiring two cost sections 
should be rescinded immediately; one cost section is 
Sufficient. Continued attempts to make the SAR "all 
things to all men“ by continual changes will result 


in it “being nothing to everyone." 
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The Congress should educate itself on the tenuous 
Mature and Uncertainty in cost estimates. To that 
end it is recommended that a cost range not a point 
estimate be included in the SAR. 

The Navy and DOD should consider some policy or 
guidance concerning engineering cost estimates. An 
expanded "Should-Cost" program offers one alternative 
to improving this area. .Another, and more drastic 
option, would be a detailed instruction or handbook 
on engineering cost estimating. Either would be a 
major departure from current practices. 

Research is recommended in the following SAR or SAR 
related areas: 

Effects of P/M rotations on SAR reporting. 

To what extent do the P/M and contractors submit 
"optimistic" cost baseline estimates either intentionally 
or due to improper techniques? 

Do contractors include inflation in their cost 
estimates which are in turn escalated by the P/M - in 
effect compounding the inflation? 

What inflation/escalation rates should be used for 
future predictions? 

What are the definitions and proper uses of inflation 
and escalation? 

Only one cost baseline estimate should be carried in 
the SAR; the PE prior to DSARC II (if applicable) then 


the DE following DSARC II. A one time change analysis 
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Should be included to explain the difference be- 
tween the two. 

The detailed variance analysis sections of the SAR 
should be eliminated to enhance the readability and 
understanding of the report while reducing its 
complexity. 

DOD should not restrict the SAR producer to an es- 
calation rate 1/3 to 1/2 of what the current economic 
inflation rate is demonstrating. This results in 
unrealistic cost estimates and increased pressure when 
the realistic indicies are applied. 

To enhance the understanding of the SAR, ASD (Comp- 
troller) should revise DOD 7000.3 for each change 

of the reporting criteria rather than sending memo- 
randa to the service secretaries. 

The NPS Library should be funded for additional sup- 
pecrmmerominc oA Student. “In particular, “Aerospace 
Daily" and current files of DOD and Navy Instructions 
should be maintained for more effective support of 
student research. This is especially true in an era 


of reduced travel funding. 
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